What Makes an Online Instructional Video Compelling?

Online courses and programs continue to expand.¹ The potential benefits of online programs often cited include the ability to reach different audiences such as nontraditional students, working professionals, and international students; the variability and flexibility of online learning tools; and the possible reduction of costs — particularly relevant given rising tuition.

Video has supported education for many years, and instructional videos are often a key component in online courses. Video has the ability to convey material through auditory and visual channels, creating a multisensory learning environment. Yet much remains unknown as to what makes compelling instructional video, especially in the online environment. What characteristics do students perceive as influencing their learning? What videos receive the most views? These questions are important in online course design, and the teaching and learning experiences. It is also important when considering that even though producing video is less complicated and less expensive than ever, media still requires production resources and therefore strategizing the best ways to allocate those resources.

The School of Continuing Education(SCE) at Columbia University has an internal team that produces online courses for a variety of programs. Instructional design lies at the core of the development of online courses, and the team carries out an ongoing examination of how design impacts learning objectives. To learn more about compelling video, we examined analytics from the video hosting platform and recruited 10 students to participate in in-depth interviews about their learning experiences with course media. This article presents some of the emerging findings from this data.

School of Continuing Education, Columbia University

The School of Continuing Education at Columbia University offers a variety of master's and certificate programs built around emerging and interdisciplinary professions in fields such as communications, technology, and sustainability. Of these programs, several are online or "hybrid" programs (where a significant amount of course activities occur online but there are also intensive face-to-face residency sessions each semester). The online courses are hosted on the learning management system Canvas by Instructure, where the majority of the materials are posted and social exchanges occur: syllabi, activities, readings, videos, discussion forums, assignments, etc. Most of the courses also have weekly "live sessions" hosted on the webinar platform Adobe Connect, where students and faculty members meet for one to two hours in virtual events that encompass lectures, group work, student presentations, and other forms of classroom interactions.

The graduate student population at SCE consists of adult learners; research has shown that adult learners embody characteristics such as self-direction, internal motivation, and goal orientation.² Many of the students are full-time working professionals with several years of experience; a significant number already hold advanced degrees on admission. The flexible schedule of online courses is often an ideal fit for those working full time and with family obligations.

Media Workflow for Online Courses

An internal team at SCE collaborates with faculty to envision and produce online courses. The instructor first begins working with an instructional designer, and then consults with other team members for media opportunities, technology needs, webinar ideas, and other course elements. When creating the media strategy, different elements are considered, such as how media can help students reach instructional objectives; how long a "shelf life" the media will have; and the amount of time/costs production will take (location shoots and animation require more resources, for instance). The formulation of the media strategy involves an active conversation between the instructional designer, media producer, and faculty member.

Once a media plan has been established, pre-production begins. Faculty members create and send outlines or scripts for review; animated videos are storyboarded. The media team arranges dates for production, and reserves spaces for video shoots. During production, audio, lighting, camera, backdrops, a teleprompter, etc. are set up; videos most often are produced with faculty members delivering the content. In the post-production process, videos are edited one to three weeks after shooting; a subsequent review process involves the instructional designer and instructor. Final videos are compressed, exported, and uploaded to Kaltura (an open-source video platform for storing and embedding media used by many higher education institutions).

The videos SCE produces have a wide range in both purpose and production value. Past projects include animated pieces, documentaries, guest speaker interviews, content lectures, simulations, scripted scenarios with actors, and other types of videos. The media team generally produces around 50–80 media assets every semester.

The following video shows an excerpt taken from a media content lecture, featuring a faculty member discussing brain stimulation. This was a location shoot filmed at a neuroscience lab, and the postproduction included selective animation.

Brain lecture excerpt showing animation (42 seconds).

Methods

To gain insight into what videos received high numbers of views (and what assets were not watched), we considered analytics from Kaltura. Kaltura tracks a number of data points, including the number of video views, the average play-through rate (how much time the average viewer watched a video), number of video downloads, devices through which videos were accessed, page impression rates, etc.

Analytics offer one dimension of user interaction with videos; to gain another perspective, 10 students were interviewed in order to understand their "lived experiences"³ and perceptions of course media. These semi-structured interviews lasted 30–45 minutes; participants discussed in broad and specific terms their viewing, sharing, and watching habits of online course videos. How do the students reflect on their experiences with course videos? Did the media help the students learn content, especially in ways that contrasted with text? Did the students watch the media to completion? With permission, the interviews were recorded, then transcribed and analyzed.

What Do Analytics Tell Us?

The hosting platform Kaltura presents numerous data points related to media, which offer indicators regarding video viewership. Kaltura tracks view numbers, player impression rates, what device and what browser viewers are accessing media, etc. An assumption behind examining the media analytics is that higher play-through rates and higher view numbers of videos are more "positive," in that students were compelled enough to watch the video in its entirety, and students watched the video many times. The play-through rate is the amount of time the average viewer watches a video. (For most online videos, the average viewer does not watch 100% of the video to completion, although this does not mean no one watches it to completion. For instance, if one viewer watched 20% of a video, and another watched 100%, the average view time would be 60%).

High View Numbers = Direct Connections to Course Assignments

Videos with high numbers of views usually have a direct connection to course assignments (or course assessments). If a video contains required content that a student must use for an assignment or discussion posting, it is correlated with higher view numbers. Two examples of videos with very high view numbers include a Pecha Kucha video used in a strategic communications course, and a Management Operating System video produced for a technology management course. These videos had view numbers that were three to five times the number of students, suggesting that most students watched the videos more than one time.

The pecha kucha presentation format emphasizes brevity and imagery (no text) in the presentation. Delivering a pecha kucha presentation to an audience of peers is a significant assignment in one of the strategic communication courses. The example of the Pecha Kucha video (which had the highest number of views of all SCE online course media in the 2012–2013 academic year) featured a student from a previous cohort delivering a model of this presentation; this student was identified by the faculty member as an exemplar. Some of the factors behind the video's high viewing numbers include:

• It is a prototype of a key course assignment.
• A discussion forum was associated with the video (and discussion posts are also tied to a participation grade).
• The video highlights how media can convey information that text or imagery alone cannot, by demonstrating presentation techniques such as timing, body language, vocal delivery, etc.

Management Operating System, a video in a technology management course, also had high view numbers; the views were more than three times the number of students enrolled. This video featured the faculty member delivering a script (using a teleprompter), with the talking-head explanation interspersed with animation. The animation visually displayed a framework students needed to use in a required course assignment; the animation and faculty description provided a visual, synthesized, and contextualized explanation of the framework (figure 1). The video also ended with a scenario that students were expected to respond to in a discussion forum.

Figure 1. A still from the Management Operating System video featuring a visual framework

Average Viewing Time = Four Minutes

A consistent finding across many courses and programs at SCE is that the average amounts of time viewers watch media (in aggregates) is approximately four minutes. This average viewing time repeats across programs and courses, even when considering longer-form videos. To clarify, this does not mean every video is watched an average of four minutes; rather, this is the average number when examining entire categories, such as all media produced in one semester or all media produced in one year.

The four-minute viewing time has influenced the instructional design and media strategy of video production. Of course, this is not to make the argument that longer videos do not belong online, especially videos with a narrative, or that four minutes is a steadfast design constraint. However, the production team has moved toward creating shorter media pieces. When producing longer-form lecture content, media is often chunked into shorter content segments. Producing 45-minute lectures that "copy the classroom onto the Internet" is not recommended.

Videos Watched on Computers

The online programs at SCE strive to be designed for "any device, anywhere," accessible by computer, tablet, or phone. These design values persist, but the analytics demonstrate that few students view course videos from a tablet or mobile device. The majority of students watch course videos on a computer. In 2013, the percentage breakdown for watching videos by device was

• 92 percent computer,
• 5 percent tablet, and
• 3 percent mobile.

In some ways this is a counterintuitive finding, especially considering the increasing penetration and prevalence of mobile devices and applications. These data suggest the context in which students approach videos is at home or an office, and not likely in public spaces (such as while commuting).

What Do Student Interviews Tell Us?

Interview participants were selected from two SCE master's degree programs: a program in communications, and another focused on technology. These programs were selected due to the divergent instructional content and the significant role of media in the course designs. The videos in these programs include screen-captured PowerPoint presentations, instructor-focused content lectures, animated videos, short documentaries, and simulations. The media in these two programs have a wide range of production values, length of time, and instructional purposes.

Instructor Presence

"The most engaging videos for me [are] when the professors use wit and humor."

Considering that one of the most significant factors of online course quality is instructor presence and interpersonal interaction,⁴ one of the benefits video can offer is creating faculty presence in an online environment. In the interviews, students cited faculty presence as a key factor related to their engagement and perceived learning from videos. Humor and wit were described positively. Participants also mentioned the benefits of adding personable context to a subject; for instance, faculty members giving examples from their professional experiences about subject material. As one student explained, "The reading is very didactic or academic, but the videos are very real-case scenarios. The instructor narrates: 'How do you take that academic learning into the real world? What does that mean when you're looking at these financial statements?'" Another participant offered: "[The videos] are better than just reading the material because it has more of that human element."

The following media excerpt from the lecture "Time Value of Money" highlights how humor and conversational delivery can foster instructor presence in media.

Humor and engaging delivery in a lecture (47 seconds).

Multimedia Presentation

"[The faculty member] was writing out the equations, and drawing arrows to them, and narrating along with it, so it was […] more narrative, more engaging, and more instructional, because even though it was finance formulas, it was a very visual thing."

A major affordance of video is the ability to produce multimedia elements and create dynamic learning artifacts. This may be self-evident, yet often instructional videos are produced without much design devoted to sound or imagery.

Students repeatedly described the audio/visual elements of video as useful aspects of online course videos. Throughout the interviews, all participants evaluated charts, graphs, photographs, and other visuals relevant to the content area in positive terms. Conversely, a couple of students voiced their dissatisfaction with videos that they did not perceive as a value-add over text (they said videos they viewed did not include useful audio/visuals and that they could have just as easily read a transcript for the same information).

A few participants discussed how visuals in course videos enhanced their learning and retention of subject content. One interviewee remarked,

"I still remember so much more of the visual presentation because there were shapes and facts and figures that comes to mind rather than a page saying a bunch of statistics. So [the video] makes it more memorable."

In another conversation, a student reflected on a series of videos that combined clips from a popular television show with a faculty member's explanation of different ethical frameworks:

"When the concept was confusing, there was a clip from the show that illustrated exactly what he was talking about. So that helped us understand it better and also for it to stick longer because we're remembering the clip versus just being told."

Production Values

"I think because the video is sophisticated there's an underlying assumption that the content we're about to watch has some level of sophistication."

The question of production values has multifaceted answers. Because almost all computers come equipped with a webcam, and smartphones have video recording capabilities, it is incredibly quick and inexpensive to produce lightweight videos. However, the user experience of watching videos with poorly recorded audio or other amateur qualities (such as shaky camera work) is arguably not a great one. On the other hand, the sky is the limit regarding video production: departments could spend millions of dollars on equipment, animation, graphics, location shoots, etc.

The interviews yielded no clear thematic finding related to production values. Generally, participants said they wanted and/or appreciated high production values. Two students mentioned they expected professional-quality videos given their tuition expenses. Others positively described videos that included green-screen effects and animation. However, two participants described as distracting the more "produced" elements like multiple camera angles. One interviewee reported that his favorite videos were lo-fi PowerPoint-narrated presentations, produced by a faculty member on her home computer, because her delivery was engaging and he was motivated to learn the content. As one student commented, "I don't think it's the production value as much as it's the content and the professor getting the point across."

Viewing Habits

"I guess one thing that I slowly learned is that I do need to treat it as a class even though I'm not there in person with the professor and students. Because at first I was like, 'oh I can just watch this video and multitask,' and then I realized: no, I actually need to treat it like a class and watch it, sit down, and take notes."

A consistent theme across the interviews is that students report their viewing habits of course media to mirror that of sitting in a class lecture. Almost all of the interviewees explained they view the course videos on their computers. Most said they take notes as they watch course videos. Some described how if a media piece features an instructor delivering a presentation and the presentation is available for download, they would download the presentation and keep it open in another window to reference as they watched the video. Note also that many participants described their viewing habits in relation to work, such as watching course videos during their lunch break or when traveling on an airplane for business trips.

While the ability to download course videos is not available in all courses, and analytics show that very few people download videos even when the feature is available, two students reported they downloaded course videos and appreciated the option. As one participant commented,

"I downloaded all of them. I figured, not only for media content or media consumption, but if I ever wanted to refer back to them, I didn't want to lose access to them. Because they were a valuable resource, and kind of a record of where I had been, so I liked it for that reason."

In this instance the student archived the course videos for possible future reference and as a memento of the course, similar to a valued textbook.

Summary

These emerging findings, taken from both quantitative and qualitative data, provide some insight as to what characteristics of online videos students describe as compelling, and what types of videos receive the most views. The developing themes have influenced the design and strategy of media production at SCE, including:

• Strategizing videos to tie directly to course assignments and/or assessment
• Advising faculty members to use conversational language in production; also encouraging them to use humor and draw on past experiences
• Adding audio/visual elements to the video that supplement the content; the videos should not convey information that students could just read as text
• Producing high-quality videos (despite mixed findings related to production values, elements such as professional sound, lighting, and graphics are considered important when creating high-quality media)
• Keeping the four-minute view time as a design consideration, especially when producing longer-form content lectures that can be broken up into shorter segments

With the advent of new technology tools and new online programs, many research questions around instructional media have begun to emerge. For instance, how might student-produced media (through lightweight tools such as cell phones or webcams) influence instruction and social interaction? In hybrid programs, how can video best supplement face-to-face sessions? How do graphic design elements (such as the video thumbnail, a video embedded on a course page, or types of text surrounding a video) influence viewing habits? Online tools and online programs continue to increase, and many opportunities exist for further investigating best practices of online instructional design.

1. Over 7.1 million postsecondary students are enrolled in at least one online course in the United States (over a third of all postsecondary students), according to a 2013 survey by the Babson Survey Research Group. (See the report "Grade Change: Tracking Online Education in the United States" by I. Elaine Allen and Jeff Seaman.)
2. Dolores Fidishun, "Andragogy and Technology: Integrating Adult Learning Theory as We Teach with Technology" (2000).
3. Irving Seidman, Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences (3rd ed.) (New York: Teachers College Press, 2006).
4. Shanna Smith Jaggars, Nikki Edgecombe, and Georgia West Stacey, "Creating an Effective Online Instructor Presence," Community College Research Center, Teachers College, Columbia University, (2013).